1. Field of the Invention
The present invention relates to a technique for generating a three-dimensional computer graphics (hereinafter, abbreviated as xe2x80x9cCGxe2x80x9d) model from an object in the real world represented in an actually shot image so that the object is reproduced in a virtual world.
2. Description of the Related Art
With recent high performance of computer systems and development of multimedia processing techniques, an environment where advanced three-dimensional computer graphics can be processed is emerging, and accelerator boards dedicated to three-dimensional graphics processing are being provided. With improvement of graphics processing capability, a virtual reality system such as a walk-through system that handles three-dimensional CG models built in a computer system is in increasing demand. In the case where a three-dimensional object is displayed in this virtual reality system, it is difficult to reproduce a three-dimensional CG model artificially created by animators so as to look as realistic as a real thing in the real world. Therefore, the approach of capturing an object in the real world and reproducing it naturally in a virtual space so as to create a more natural three-dimensional object is under development. In other words, a method for generating a three-dimensional CG model by taking an actually captured image into a computer and processing the captured image with an image processing technique is under development.
A three-dimensional CG model of an object represented in an actually captured image cannot entirely be generated by automatic processing, as far as the current image processing technique is concerned. It is necessary for the user to perform manual input assistance. In the prior art, there are various methods for input assistance.
A first method, most generally, is as follows. A plurality of images are prepared by photographing an object from various viewpoints, and the same part of the object in these images is specified as a corresponding part. Generally, this is referred to as a stereo method.
However, in the first method, it is difficult to specify the corresponding points fully automatically. In addition, when there are a large number of corresponding points, it is troublesome for the user to specify the corresponding points manually.
A second method for creating a model is as follows. A previously prepared basic model is superimposed on an actually captured image, and the basic model is manipulated for modification and adjustment so as to take on the same posture as that of the object represented in the actually captured image. A desired CG model can be obtained by this basic model manipulation.
However, in the second method, manipulation such as movement or rotation is required in order that the model takes on the same posture as that of the object represented in the actually captured image. This manipulation requires skills.
The approach of matching a model to an object represented in an actually captured image automatically has been researched. However, it is necessary to prepare an initial state where the object in the actually captured image and the basic geometric model have the same posture to some extent before automatic operation. Therefore, although automatic operation is used for the final fine adjustment of the input by the first and second methods, the problems described above in conjunction with the first and second methods are still to be solved.
Therefore, with the foregoing in mind, it is an object of the present invention to provide a three-dimensional CG model generator that allows the input assistance by the user for matching between an actually captured image supplied thereto and a basic model to be simplified and can generate a three-dimensional CG model without special skills.
Furthermore, it is another object of the present invention to provide a three-dimensional CG model generator that can generate a three-dimensional CG model that represents the supplied actually captured image more precisely.
Furthermore, it is another object of the present invention to provide processing steps for realizing the three-dimensional CG model generator of the present invention using a computer in the form of a computer-readable recording medium.
In order to achieve the above objects, a three-dimensional CG model generator of the present invention is an apparatus for generating a three-dimensional CG model of a three-dimensional object represented in a two-dimensional image, based on information from the two-dimensional image representing the three-dimensional object supplied thereto, includes an image inputting part for receiving image data of the object taken from a certain viewpoint; a basic pose information storing part for storing pose information on all patterns of a basic geometric model as basic pose information, the pose information indicating a pose with relationships of vertexes and edges of the image; a pose information designating part for acquiring the pose information by designating a vertex and an edge of the object represented in the supplied image; a pose matching part for performing pose matching by comparing the pose information acquired by the pose information designating part and the basic pose information stored in the basic pose information storing part, and specifying a basic geometric model having corresponding pose information and a corresponding pose; and an automatic fitting part for deforming and adjusting the pose of the basic geometric model specified by the pose matching part so as to conform to the pose of the supplied image. The three-dimensional CG model of the object is generated based on the pose information of the supplied image.
With this embodiment, the characteristics of the object in the supplied actually captured image can be extracted using information on basic elements of a vertex and an edge and information on the correlation between these elements as pose information, and a corresponding basic geometric model can be selected by pose matching with the basic geometric model.
Next, it is preferable that the three-dimensional CG model generator of the present invention includes a pose matching history information storing part for storing history information of the results of the past pose matching, and that the basic geometric model candidate displaying and selecting part displays a plurality of candidates with priorities that are assigned based on the pose matching history information when the plurality of candidates are displayed.
This embodiment allows the three-dimensional CG model generator to include a learning function. Therefore, even if a plurality of candidates are obtained as a result of pose matching, the candidates with priorities that are assigned based on the past history can be presented to the user, so that the selection process by the user can be simplified.
Herein, the priorities can be assigned, for example based on how many times the candidate is selected as a matched basic geometric model in the past, or based on how recent the candidate was last selected as a basic geometric model (least recent used).
Next, it is preferable that in the process of designating vertexes and edges of the supplied image sequentially by inputting the vertexes and the edge from the pose information designating part, the pose matching part includes a partial pose matching part for finding a match between information on a portion of a pose of the basic geometric model stored in the basic pose information storing part and information based on a portion of a pose that is constituted by the designated and input vertexes and edges; and a basic geometric model candidate displaying and selecting part for presenting a corresponding basic geometric model obtained as a result of the partial pose matching by the partial pose matching part. It is preferable that the results of the partial pose matching are updated and presented in response to information on a vertex and an edge of the supplied image being input from the pose information designating part.
This embodiments makes it possible to perform partial pose matching based on the partial pose information, which is the pose information on an already input portion of the entire pose, every time information on a vertex or an edge is input in the process of sequentially designating the vertexes and the edges of the image supplied by the user from the pose information designating part, and to present dynamically the candidates of the basic geometric model in the range that can be limited at the point.
Next, it is preferable that the three-dimensional CG model generator further includes a basic pose extracting part for receiving a three-dimensional CG model as a basic geometric model, generating a two-dimensional image of the basic geometric model which is a projected image from one viewpoint, extracting pose information from the two-dimensional image, changing the viewpoint, repeating the extraction of the pose information so that all patterns of poses that the basic geometric model can take are extracted.
With this embodiment, the basic pose information to be registered in the basic pose information storing part can be generated automatically.
Furthermore, the three-dimensional CG model generator of the present invention can be configured by a computer based platform by installing a processing program that can provide processing steps for realizing the above-mentioned three-dimensional CG model generator of the present invention.
These and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.